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A  risk-driven  contract  structure  is  proposed  to  enhance 
the  cost  realism  of  competitive  proposals  for  the  Engi¬ 
neering  and  Manufacturing  Development  (EMD)  phase  of 
the  acquisition  life  cycle.  The  authors  employ  an  economic 
theory  framework  to  discuss  how  cost-plus  contracts 
typically  used  during  this  phase  have  inadvertently 
reinforced  the  sources  of  contractor  and  government 
optimism  bias.  By  mapping  probabilistic  cost  estimates 
to  profit  distributions,  risk-driven  contracts  offer  a  struc¬ 
tured  method  to  expose  contractors  to  more  cost  risk 
during  EMD.  Holding  contractors  accountable  for  their 
cost  estimates  and  cost  performance  should  enhance 
the  realism  of  cost  proposals,  limit  the  government’s 
ability  to  commit  to  too  many  programs,  and  reduce 
the  cost  growth  that  continues  to  plague  the  defense 
acquisition  system. 
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The  Government  Accountability  Office  (GAO)  reported  a  combined 
$296  billion  in  cost  growth  on  the  Department  of  Defense’s  96  major 
acquisition  programs  in  fiscal  year  (FY)  2008.  Sixty-nine  percent  (64  of 
the  96  programs)  experienced  cost  growth,  demonstrating  that  the  cost 
growth  is  not  just  limited  to  a  few  programs.  In  addition,  42  percent  (40 
programs)  reported  at  least  25  percent  unit  cost  growth,  demonstrat¬ 
ing  that  the  bulk  of  the  growth  is  not  limited  to  a  few  programs  either. 
Finally,  75  percent  (69  programs)  experienced  increases  in  research, 
development,  test,  and  evaluation  (RDT&E)  costs,  demonstrating  that 
problems  often  start  early  in  the  acquisition  life  cycle  (GAO,  2009,  p. 
2).  This  last  statistic  is  particularly  important  to  this  research  since 
risk-driven  contracts  are  targeted  at  improving  cost  realism  for  system 
development  efforts. 

To  put  this  $296  billion  cost  growth  into  perspective,  consider  that 
the  FY  2012  President’s  Budget  Request  is  $671  billion  (including  funding 
for  the  operations  in  Afghanistan  and  Iraq),  with  $204  billion  allocated 
to  acquisitions  ($128  billion  for  procurement  and  $76  billion  for  RDT&E) 
(DoD,  2011,  p.  8-3).  Thus,  ifDoD  still  wants  these  96  weapon  systems,  it 
must  cover  an  unfunded  liability  greater  than  its  annual  acquisitions 
budget.  This  daunting  task  is  compounded  by  the  current  state  of  the 
economy  and  the  resulting  fiscal  pressures.  Defense  Secretary  Robert 
M.  Gates  (2011)  remarked: 

This  department  simply  cannot  risk  continuing  down  the  same 
path-where  our  investment  priorities,  bureaucratic  habits,  and 
lax  attitudes  towards  costs  are  increasingly  divorced  from  the 
real  threats  of  today,  the  growing  perils  of  tomorrow,  and  the 
nation’s  grim  financial  outlook. 

In  support  of  enhancing  cost  realism,  this  article  is  organized  into 
three  parts:  (a)  a  brief  review  of  the  difference  between  cost  growth  and 
cost  overruns,  (b)  a  discussion  of  the  primary  reasons  for  unrealistic 
cost  estimates,  and  (c)  a  detailed  demonstration  of  risk-driven  contracts. 

Cost  Growth  vs.  Cost  Overruns 

Cost  growth  implies  an  increase  in  the  life-cycle  cost  estimate,  which 
may  or  may  not  affect  the  cost  performance  of  the  current  contract.  For 
example,  a  choice  to  use  a  specific  material  during  system  development 
could  lead  to  increased  procurement  costs  without  necessarily  increas- 
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ing  the  development  costs.  On  the  other  hand,  a  cost  overrun  results 
when  a  program  exceeds  the  target  cost  of  its  contract,  which  usually 
leads  to  life-cycle  cost  growth  despite  the  prospect  for  future  efficiencies. 

When  target  costs  are  unrealistic,  overruns  do  not  necessarily  indi¬ 
cate  excessive  expenditures  (Cummins,  1977,  p.  179).  Despite  the  reasons 
for  overruns,  they  are  almost  always  counterproductive.  First,  they  often 
lead  to  funding  instability  within  a  portfolio,  which  in  turn  leads  to 
adjustments  between  programs  (damaging  healthy  programs  to  rescue 
sick  ones),  reductions  in  requirements  or  procurement  quantities,  or 
extensions  to  schedules  (GAO,  2008,  p.  11).  Second,  overruns  can  damage 
public  perception  and,  as  a  result,  diminish  congressional  support  and 
risk  eventual  cancellation  (Cummins,  1977,  p.  179).  And  third,  overruns 
can  be  perceived  as  a  managerial  failure  and  lead  to  drastic  personnel 
replacements  in  the  government  and  contractor  program  offices  (Scherer, 
1964,  pp.  275-276). 

Reasons  for  Unrealistic  Cost  Estimates 

Cost  estimates  can  be  unrealistic  for  a  multitude  of  reasons,  which 
include  an  overemphasis  on  the  technical  cost  drivers,  optimism  bias, 
and  misaligned  contract  incentives. 

Overemphasis  on  Technical  Cost  Drivers 

While  room  for  improvement  always  exists,  today’s  professional 
cost  estimators  have  an  abundance  of  tools  from  which  to  leverage  best 
practices.  Sophisticated  cost-estimation  guides  have  been  published 
by  the  Army,  Navy,  Air  Force,  National  Aeronautics  and  Space  Admin¬ 
istration  (NASA),  GAO,  RAND,  International  Society  of  Parametric 
Analysts/Society  of  Cost  Estimating  and  Analysis  (ISPA/SCEA),  and 
the  Space  Systems  Cost  Analysis  Group  (SSCAG).  Also  available  are 
extensive  articles,  conferences,  and  training  and  certification  oppor¬ 
tunities  from  professional  societies  like  ISPA,  SCEA,  SSCAG,  and  the 
United  Kingdom’s  Society  of  Cost  Analysis  and  Forecasting  (SCAF). 
In  addition,  Garvey  (2000)  authored  the  definitive  textbook  on  cost 
estimation  wherein  he  describes  the  principal  methods  for  address¬ 
ing  cost  uncertainty.  Finally,  a  vast  array  of  software  tools  can  be  used 
to  construct  cost  estimates,  such  as  the  Automated  Cost  Estimating 
Integrated  Tools  (ACEIT),  Crystal  Ball,  @RISK,  PRICE,  System  Evalu¬ 
ation  and  Estimation  of  Resources  (SEER),  NASA/Air  Force  Cost  Model 
(NAFCOM),  Constructive  Cost  Model  (COCOMO)  II,  and  Constructive 
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Systems  Engineering  Cost  Model  (COSYSMO).  In  an  unbiased  world, 
subject  matter  experts  applying  these  tools  and  best  practices  would 
generate  more  accurate  and  reliable  cost  estimates.  But  the  problem  is 
not  a  lack  of  guidance  or  tools— it  is  that  the  cost  estimation  community 
usually  considers  only  the  technical  variables  contributing  to  cost  risk. 

Optimism  Bias 

An  understated  cause  of  cost  overruns  is  optimism  bias,  which  is 
defined  as  the  tendency  for  people  to  be  overconfident  in  their  predic¬ 
tions  (Valerdi  &  Blackburn,  2009).  A  common  form  of  optimism  bias  is 
optimistic  technical  estimates,  which  range  from  the  weight  of  a  hard¬ 
ware  component  to  the  number  of  software  lines  of  code.  Perhaps  the 
most  difficult  and  subjective  part  of  cost  estimation  is  eliciting  these 
estimates  from  technical  experts.  Unfortunately,  it  has  been  shown  that 
most  experts  are  overly  optimistic  in  providing  both  their  most  likely 
and  worst-case  estimates  (Russo  &  Schoemaker,  1992).  Hubbard  (2010, 
pp.  57-77),  building  on  the  original  research  of  Brier  (1950),  provides  a 
practical  technique  to  “calibrate”  experts  to  provide  better  estimates 
when  confronted  with  uncertainty. 
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A  second,  and  equally  damaging,  form  of  optimism  bias  is  optimistic 
management  estimates  by  both  contractors  and  the  government.  The 
contractor’s  optimism  bias  is  caused  by  pressures  to  win  competitions. 
William  M.  Allen,  Boeing’s  president  in  1964,  admitted,  “I  can  think 
of  a  lot  of  programs  in  the  Boeing  Company  where,  if  the  estimate  had 
been  realistic,  you  wouldn’t  have  had  the  program.  And  that  is  the  truth” 
(Butts  &  Linton,  2009,  p.  36). 

While  two  or  more  contractors  are  often  funded  during  early  tech¬ 
nology  development  and  prototyping  efforts,  the  government  typically 
only  funds  a  single  contractor  during  EMD  due  to  prohibitively  high 
system  development  costs.  After  several  years  of  focused  government 
investment,  the  incumbent  contractor  normally  develops  a  significant 
technical  advantage.  Thus,  the  government’s  options  are  greatly  limited 
since  the  prospect  of  reattempted  competition  is  dubious  at  best.  As  a 
result,  the  contractor  that  wins  the  competitive  EMD  downselection 
usually  monopolizes  the  production  and  sustainment  efforts  as  well. 
With  so  much  long-term  revenue  and  profit  on  the  line,  competition  to 
win  the  EMD  contract  is  intense.  And  since  cost  is  a  leading  variable  in 
the  government’s  source  selection,  there  is  a  strong  motivation  to  provide 
the  lowest  cost  proposal. 

The  government’s  optimism  bias  is  caused  by  the  Services’  desire 
to  secure  funding  for  new  programs  and  sustain  funding  for  existing 
ones.  To  maintain  the  appearance  of  affordability,  cost  estimates  that 
fit  within  authorized  budgets  are  at  least  tacitly  encouraged  (William¬ 
son,  1967,  p.  229;  GAO,  2008,  pp.  20-21).  In  addition,  U.S.  Senators  and 
Representatives  often  contribute  to  the  government’s  optimism  bias 
by  supporting  programs  with  poor  business  cases  when  the  funding  is 
allocated  to  their  constituents. 

Misaligned  Contract  Incentives 

While  strong  leadership  and  accountability  may  help  reduce  opti¬ 
mism  bias  amongst  stakeholders,  properly  implemented  contract 
incentives  are  an  even  stronger  antidote.  Figure  1  organizes  the  most 
prevalent  contract  types  by  their  degree  of  risk  sharing  and  typical  use 
throughout  the  acquisition  life  cycle.  Cost  Plus  Fixed  Fee  (CPFF)  and 
Firm  Fixed  Price  (FFP)  contracts  represent  two  polar  extremes  with  no 
risk  sharing.  The  government  assumes  all  cost  risk  in  a  CPFF  contract, 
and  the  contractor  assumes  all  cost  risk  in  an  FFP  contract.  Cost  Plus 
Incentive  Fee  (CPIF)  and  Fixed  Price  Incentive  Firm  Target  (FPIF) 
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contracts  offer  a  middle  ground  with  risk  sharing  by  both  the  government 
and  contractor.  Of  these  two  incentive  contracts,  only  FPIF  contracts 
expose  contractors  to  a  potential  loss,  but  as  with  FFP  contracts,  maxi¬ 
mum  losses  are  not  constrained.  Theoretically,  a  contractor  can  be  forced 
into  bankruptcy  in  attempting  to  fulfill  the  requirements  of  an  FFP  con¬ 
tract.  However,  with  the  dwindling  defense  industrial  base  (Aerospace 
Industries  Association,  2009),  it  is  not  in  the  government’s  best  interest 
to  force  a  contractor  out  of  business.  In  addition,  contractors  are  likely 
to  mount  protracted  legal  battles  to  protect  their  interests,  which  are 
counterproductive  in  delivering  capability  to  the  warfighter  and  a  poor 
use  of  taxpayer  resources. 

On  the  other  hand,  a  contractor’s  maximum  liability  for  overrun¬ 
ning  a  typical  CPIF  contract  is  no  profit.  While  their  short-term  stock 
prices  may  be  impacted,  at  least  four  reasons  can  be  set  forth  to  explain 
why  contractors  still  benefit  when  they  receive  no  profit  (Fox,  1974,  pp. 
242-243): 

•  Scientists  and  engineers  are  gainfully  employed  (or  hired) 
and  available  for  future  programs. 

•  Technology  competency  is  accrued,  which  improves  their  mar¬ 
ket  position  for  future  government  and  commercial  business. 

•  Facilities  and  equipment  are  maintained  and  often 
upgraded  at  the  government’s  expense. 

•  Overhead  expenses  for  other  programs  (and  potential  new 
programs)  are  slightly  reduced  by  contributions  to  the  over¬ 
head  pool. 
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FIGURE  1.  RECOMMENDED  CONTRACT  TYPES  FOR  EACH 
ACQUISITION  PHASE 
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Note.  IOC  =  Initial  Operational  Capability;  PDR  =  Preliminary  Design  Review;  LRIP  =  Low 
Rate  initial  Production;  IOT&E  =  Initial  Operational  Test  and  Evaluation;  FRP  =  Full  Rate 
Production;  CPFF  =  Cost  Plus  Fixed  Fee;  CPAF  =  Cost  Plus  Award  Fee;  CPIF  =  Cost  Plus 
Incentive  Fee;  FP/F  =  Fixed  Price  Contract  with  Incentive  Firm  Target;  FFP  =  Firm  Fixed 
Price.  Adapted  from  Operation  of  the  Defense  Acquisition  System,  DoD  Instruction 
5000.02,  2008,  p.  12. 


Properly  designed  incentive  contracts  address  classic  moral  haz¬ 
ard  and  adverse  selection  problems  (McAfee  &  McMillan,  1986,  p. 
326).  Moral  hazard  is  the  propensity  to  act  differently  when  insulated 
from  the  risk  of  a  loss.  Thus,  moral  hazard  encompasses  the  propen¬ 
sity  for  contractors  to  underestimate  competitive  program  costs  and 
carry  excess  organizational  slack  during  contract  execution  when  not 
exposed  to  a  potential  loss.  Organizational  slack  is  characterized  by 
inefficiently  high  operating  and  investment  expenses  (Williamson,  1967, 
pp.  224-226).  Operating  expenses  can  be  reduced  through  the  adoption 
of  lean  practices  if  risk  sharing  is  high  enough  to  overcome  the  cultural 
barriers  to  change.  In  addition,  contractors  are  likely  to  allocate  their 
best  people  to  the  contracts  with  the  largest  potential  losses,  which  can 
also  help  reduce  operating  costs.  Conversely,  less  risk  sharing  is  likely 
to  increase  organizational  slack  in  favor  of  more  investment  expenses. 
For  example,  Scherer  (1964,  p.  263)  identifies  the  government’s  source 
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selection  emphasis  on  the  availability  of  skilled  manpower  as  an  encour¬ 
aging  factor  in  contractors  maintaining  their  workforces  at  inefficiently 
high  levels. 

Adverse  selection  deals  with  the  government’s  imperfect  knowledge 
of  the  expected  cost  of  each  contractor.  Williamson  (1967,  p.  230)  boldly 
states,  “It  is  unquestionably  true  that  the  government  suffers  from  an 
information  disadvantage.”  Indeed,  contractors  benefit  from  locally 
calibrated  parametric  cost  models,  employ  the  technicians  and  engineers 
who  will  be  working  on  the  contract,  and  have  close  relationships  with 
key  suppliers. 

If  the  government  had  perfect  information  (and  was  free  from  con¬ 
tractors’  moral  hazard),  it  would  award  a  CPFF  contract  to  what  it  knew 
to  be  the  lowest  cost  contractor  to  avoid  the  risk  premium  of  incentive 
contracts  (Samuelson,  1986,  p.  1,539).  However,  since  the  government 
does  not  have  perfect  information  and  cannot  avoid  contractors’  moral 
hazard,  economists  reject  using  cost-plus  contracts  for  competitive 
source  selections  (McAfee  &  McMillan,  1986,  p.  327).  Instead,  econo¬ 
mists  advocate  contracts  that  expose  contractors  to  a  potential  loss 
to  solicit  their  unbiased  cost  estimates,  but  for  system  development 
efforts  with  high  uncertainty,  potential  contractor  losses  need  to  be 
appropriately  limited.  Otherwise,  to  avoid  the  extremely  high  cost  risks 
of  fixed-price  arrangements,  contractors  may  choose  not  to  bid,  which 
would  in  turn  reduce  the  competition  essential  to  both  guarding  against 
overestimation  bias  and  producing  viable  warfighter  options. 

As  with  the  cyclic  nature  of  most  acquisition  reforms,  DoD  has  oscil¬ 
lated  back  and  forth  between  its  preference  for  cost-plus  and  fixed-price 
contracts.  Cancian  (1995,  pp.  195-196)  traced  the  history  of  this  oscilla¬ 
tion  over  the  past  several  decades.  In  the  1950s,  he  noted  that  cost-plus 
contracts  were  the  norm.  The  resulting  huge  overruns  led  to  a  prefer¬ 
ence  for  fixed-price  Total  Package  Procurement  contracts  in  the  1960s. 
When  this  practice  failed  due  to  the  high  risks  contractors  were  forced  to 
assume,  cost-plus  contracts  resumed  their  prevalence  in  the  1970s.  Amid 
perceived  procurement  “scandals,”  DoD  again  shifted  its  preference  back 
to  fixed-price  contracts  in  the  1980s.  Of  course  this  policy  failed  again  for 
the  same  reasons,  bringing  the  defense  acquisition  process  to  its  current 
phase  where  cost-plus  contracts  are  again  dominant. 
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It  appears  the  pendulum  may  be  swinging  back  to  fixed-price  con¬ 
tracts  with  recent  directives  published  by  a  former  Under  Secretary  of 
Defense  for  Acquisition,  Technology  and  Logistics  (USD[AT&L])  (Carter, 
2010,  p.  6).  However,  the  guidance  on  using  FPIF  contracts  focuses  on 
early  production  contracts  (just  after  Milestone  C  in  Figure  1.).  This 
guidance  is  a  step  in  the  right  direction  away  from  the  subjective  Cost 
Plus  Award  Fee  (CPAF)  contracts  that  have  recently  become  common 
during  early  production,  but  does  not  address  the  misaligned  incentive 
structures  typically  used  during  system  development  when  the  cost 
uncertainty  is  even  higher. 

Risk-Driven  Contracts 

Rather  than  continuing  to  oscillate  back  and  forth  between  cost- 
plus  and  fixed-price  contracts,  DoD  could  benefit  from  embracing  a 
hybrid,  risk-driven  contract  type  for  system  development.  As  discussed 
above,  FPIF  contracts  are  inappropriate  since  they  do  not  constrain  the 
maximum  loss  potential  for  contractors.  CPIF  contracts  could  be  used 
to  expose  contractors  to  a  limited  loss  potential  by  extending  the  sharing 
line  into  the  negative  fee  region,  but  in  practice  this  is  rarely  done  since 
negotiating  an  arbitrary  maximum  cost  point  is  extremely  difficult.  For 
example,  if  a  contractor  submits  a  point  cost  estimate  of  $100  million 
with  no  further  information,  how  should  the  maximum  cost  point  be 
determined?  This  process  is  difficult  enough  when  the  minimum  fee  is 
positive.  Negotiating  an  arbitrary  maximum  cost  point  when  a  $20  mil¬ 
lion  loss  is  at  stake  could  be  unworkable. 

Notional  Probabilistic  Cost  Estimates 

By  taking  advantage  of  modern  probabilistic  cost  estimates,  risk- 
driven  contracts  provide  a  structured  method  to  impose  a  limited  loss 
potential  on  contractors.  Experience  has  shown  that  defense  acquisi¬ 
tion  program  cost  estimates  are  often  best  modeled  by  the  lognormal 
probability  distribution  because  its  right  skew  accurately  reflects  the 
disproportionate  chance  and  magnitude  of  cost  overruns  (Department 
of  the  Air  Force,  2007,  p.  96). 
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FIGURE  2.  NOTIONAL  PROBABILITY  DISTRIBUTION  FUNCTIONS 

Blue  Mean  =  100  ($M),  Blue  Variance  =  500  ($M)2 
Red  Mean  =  100  ($M),  Red  Variance  =  2,500  ($M)2 


Two  lognormal  probability  distributions  will  be  used  throughout 
this  paper  to  describe  the  risk-driven  contract  structure.  Figure  2  shows 
the  probability  distribution  functions  (PDF)  of  “blue”  and  “red”  proba¬ 
bilistic  cost  estimates  with  the  same  mean  but  difference  variances. 
The  blue  cost  estimate  represents  a  notional  Low-Rate  Initial  Produc¬ 
tion  (LRIP)  proposal,  and  the  red  cost  estimate  represents  a  notional 
EMD  proposal.  Note  that  the  red  estimate  has  both  a  higher  cost  risk 
and  opportunity  than  the  blue  estimate,  as  shown  by  its  longer  right 
and  left-hand  tails,  respectively.  With  less  of  the  design  locked  down, 
decisions  made  on  the  red  EMD  program  often  have  a  larger  marginal 
cost  impact  than  the  relatively  minor  decisions  still  pending  on  the  blue 
LRIP  program. 
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FIGURE  3.  NOTIONAL  CUMULATIVE  DISTRIBUTION  FUNCTIONS 


Blue  Mean  =  100  ($M),  Blue  Variance  =  500  ($M)2 
Red  Mean  =  100  ($M),  Red  Variance  =  2,500  ($M)2 


Figure  3  shows  the  corresponding  cumulative  probability  dis¬ 
tribution  functions  (CDF)  which  reveal  the  confidence  level  of  each 
possible  cost  from  the  notional  PDFs.  For  example,  there  is  an  80  per¬ 
cent  chance  that  the  red  program  will  cost  $133.1  million  or  less.  Table 
1  lists  selected  confidence  levels  from  Figure  3  that  are  used  in  this 
article.  Finally,  for  the  purposes  of  this  discussion,  the  blue  and  red  cost 
estimates  are  assumed  to  be  accurate  and  unbiased.  They  bound  the 
possible  costs  without  the  influence  of  any  technical  estimation  errors 
or  optimistic  biases. 


Defense  ARJ,  April  2012,  Vol.  19  No.  2:133-160 


144 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


TABLE  1.  SELECTED  CONFIDENCE  LEVELS  FROM  FIGURE  3 


Cost  ($M)  Confidence 


Blue 

Red 

65.0 

25% 

84.1 

25% 

89.4 

50% 

97.6 

50% 

100 

54.4% 

59.3% 

117.5 

80% 

120 

82.5% 

73.3% 

133.1 

80% 

140.3 

95% 

163.1 

99% 

194.5 

95% 

268.4 

99% 

Fixed  Price  Incentive  Firm  Target  Contract  Structure 

Before  describing  the  risk-driven  contract  structure,  the  expected 
profits  from  an  FPIF  contract  will  be  briefly  outlined  for  comparison 
purposes.  Consider  the  FPIF  contract  structure  shown  in  Figure  4.  The 
solid  magenta  profit  sharing  line  is  applied  to  both  the  blue  and  red  cost 
estimates  portrayed  on  the  right  “Probability”  axis.  The  target  cost  is 
set  to  $100  million— the  expected  cost  of  both  the  blue  and  red  programs. 
A  $12  million  target  profit  is  set  for  illustrative  purposes.  Finally,  a 
50/50  sharing  ratio  and  120  percent  ceiling  are  set  in  accordance  with 
USD(AT&L)’s  recommended  point  of  departure  (Carter,  2010,  p.  6).  The 
point  of  total  assumption  (PTA)  cost  and  profit  ($116  million  and  $4  mil¬ 
lion,  respectively)  are  calculated  based  on  the  above  variables. 
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FIGURE  4.  FIXED  PRICE  INCENTIVE  FIRM  TARGET  CONTRACT 
STRUCTURE 

Target  Cost  =  $100M,  Target  Profit  =  $12M 
50/50  Sharing  Ratio,  120%  Ceiling 

0.02 
0.016 
0.012 
0.008 
0.004 
0 


50  100  150  200  250 

COST  ($M) 

Note.  PTA  =  Point  of  Total  Assumption. 


The  expected  profit  of  each  program  is  determined  by  multiplying 
the  profit  at  each  cost  by  its  corresponding  probability  and  then  sum¬ 
ming  all  possibilities.  Thus,  the  blue  and  red  cost  estimates  are  seen 
as  weighting  functions  on  the  magenta  sharing  line.  The  net  result  is 
$10.9  million  for  the  blue  program  and  $7.5  million  for  the  red  program. 
Since  the  expected  profits  are  different  for  each  program,  this  contract 
structure  is  not  universally  applicable  to  all  cost  estimates.  To  match 
the  expected  profits  for  both  cost  estimates,  a  trial-and-error  method 
adjusting  the  sharing  ratios  and  ceiling  percentages  would  be  required. 
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FIGURE  5.  PROBABILITY  DOMAIN  REPRESENTATION 
OF  FIGURE  4 


Next,  observing  from  Figure  3  that  each  cost  has  a  corresponding 
confidence  level,  it  is  possible  to  display  the  profit  sharing  relationships 
in  the  probability  domain,  as  shown  in  Figure  5.  The  blue  and  red  cost 
estimates  each  have  distinct  profit  sharing  curves.  As  previously  dis¬ 
cussed,  the  red  program  is  seen  to  have  a  higher  profit  opportunity,  but 
also  a  much  higher  potential  loss.  Assuming  the  cost  estimates  accu¬ 
rately  bound  the  possible  costs  (and  setting  the  maximum  costs  to  the 
99  percent  confidence  levels),  the  maximum  loss  is  $43.1  million  for  the 
blue  program  and  $148.4  million  for  the  red  program.  It  must  be  noted, 
however,  that  there  is  only  a  1  percent  chance  of  incurring  these  maxi¬ 
mum  losses.  At  this  point,  it  should  be  obvious  that  this  FPIF  contract 
structure  favors  the  blue  cost  estimate.  While  contractors  might  agree  to 
this  FPIF  contract  for  the  blue  program,  it  is  highly  unlikely  they  would 
expose  themselves  to  a  $148.4  million  loss  on  the  red  program  even  when 
there  is  a  $7.5  million  expected  profit. 
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Risk  Aversion  in  Human  Decision  Making 

Economists  have  studied  the  risk  aversion  propensity  of  contractors 
to  sacrifice  higher  expected  profit  margins  in  order  to  minimize  their 
share  of  potential  losses  when  faced  with  uncertainty.  Scherer  (1964, 
p.  276)  collected  strong  empirical  evidence  to  support  this  violation  of 
expected  profit  maximization  theory  whereby  risk-neutral  contractors 
would  prefer  the  contract  offering  the  highest  expected  profit  despite  its 
potential  losses.  In  addition,  Kahnemanwonthe  Nobel  Prize  in  Econom¬ 
ics  for  modeling  the  psychology  of  decision  making  under  uncertainty. 
Working  together  with  Tversky,  Kahneman  (1984)  confirmed  that  it  is 
human  nature  to  be  risk  averse.  Their  findings  support  the  conclusion 
that  in  general  people  are  more  likely  to  settle  for  a  sure  gain  than  gamble 
for  a  higher  expected  gain.  For  example,  most  people  would  rather  settle 
for  an  $800  sure  gain  than  bet  on  an  85  percent  chance  to  win  $1,000 
(with  a  15  percent  chance  to  win  nothing)  even  though  the  latter  has  the 
higher  mathematical  expectation  of  $850  (Kahneman  &  Tversky,  1984, 
p.  341). 

Risk-Driven  Contract  Structure 

It  should  be  no  surprise  that  the  FPIF  example  cited  previously 
favors  the  blue  cost  estimate,  which  is  more  representative  of  an  LRIP 
program.  In  addition,  the  very  large  potential  loss  for  the  red  program 
confirms  why  FPIF  contracts  are  not  typically  appropriate  for  system 
development  efforts  during  EMD.  However,  rather  than  settling  for  a 
cost-plus  contract  variant  during  EMD,  government  acquisition  officials 
could  benefit  from  considering  a  risk-driven  contract. 

Unlike  the  FPIF  contract  structure,  which  draws  sharing  lines  in  the 
cost  domain,  the  risk-driven  contract  structure  starts  in  the  probability 
domain,  as  shown  in  Figure  6.  This  illustrative  contract  is  structured  by 
setting  four  profit  points: 

•  Profit  (p25)  =  $20M 

•  Profit  (p50)  =  $12M 

•  Profit  (p80)  =  $0M 

•  Profit  (p95)  =  -$20M 
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For  example,  the  target  profit  is  set  to  $12  million  at  both  the  blue  and 
red  50  percent  confidence  levels.  More  importantly,  notice  how  determin¬ 
ing  the  zero  and  $20  million  loss  levels  in  the  probability  domain  provides 
a  structured  approach  to  holding  contractors  accountable  for  overly 
optimistic  cost  estimates  or  poor  cost  performance.  The  sharing  lines 
simply  connect  (or  extend)  the  profit  points,  and  are  again  magenta  since 
they  apply  to  both  the  blue  and  red  cost  estimates. 

FIGURE  6.  RISK-DRIVEN  CONTRACT  STRUCTURE 

Profit  (p25)  =  $20M,  Profit  (p50)  =  $12M 


Blue  &  Red 
p99 

By  determining  profits  in  the  probability  domain,  risk-driven  con¬ 
tracts  reward  (or  penalize)  contractors  equally  for  equivalent  cost  savings 
effort.  For  example,  reducing  costs  from  the  50  to  45  percent  confidence 
level  earns  the  same  profit  increase  for  both  the  blue  and  red  programs. 
Thus,  risk-driven  contracts  normalize  the  relative  value  of  decisions 
made  on  programs  with  different  cost  uncertainties.  This  is  contrasted 
with  the  FPIF  contract  structure  where  saving  the  same  dollar  amount 
on  either  the  blue  or  red  program  always  earns  the  same  profit  increase 
regardless  of  the  amount  of  effort  required  to  achieve  the  savings. 
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Under  the  risk-driven  contract  structure  shown  in  Figure  6,  the 
expected  profit  for  both  the  blue  and  red  programs  is  $9.5  million.  Note 
that  there  is  no  need  to  adjust  sharing  ratios  or  ceiling  percentages  to 
achieve  the  same  expected  profit  as  described  above  for  FPIF  contracts. 
In  this  way,  risk-driven  contracts  could  provide  a  more  universal  point  of 
departure  for  EMD  contracts.  Policymakers  would  simply  have  to  deter¬ 
mine  a  few  profit  points  in  the  probability  domain  as  outlined  above. 

Figure  6  also  reveals  the  same  maximum  loss  for  both  the  blue  and 
red  programs.  There  is  a  one  percent  chance  that  either  program  might 
incur  a  $25.3  million  loss.  Further,  there  is  only  a  20  percent  chance  of 
incurring  any  loss.  Again,  while  the  goal  is  not  to  set  any  specific  profit  or 
loss  policies,  it  should  be  noted  how  the  risk-driven  contract  provides  a 
method  to  more  reasonably  limit  the  potential  losses  of  contractors  engag¬ 
ing  in  risky  development  efforts.  The  objective  is  to  set  the  loss 
probability  and  magnitude  to  the  lowest  possible  levels  that  will  counter¬ 
act  the  previously  described  moral  hazard  and  adverse  selection  problems. 


FIGURE  7.  COST  DOMAIN  REPRESENTATION  OF  FIGURE  6 
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It  is  also  instructive  to  examine  the  risk-driven  contract  structure 
in  the  cost  domain,  as  shown  in  Figure  7.  The  first  major  observation  is 
the  upper  end  of  the  red  program’s  profit  is  now  less  than  that  of  the  blue 
program  unlike  the  FPIF  contract  example  shown  in  Figure  5.  The  gov¬ 
ernment  shares  a  larger  portion  of  the  red  program  contractor’s  upside 
profit  in  return  for  limiting  its  potential  losses.  In  effect,  the  contractor 
trades  slightly  less  profit  opportunity  for  greatly  reduced  loss  risk,  which 
should  be  an  acceptable  trade  for  a  risk-averse  contractor.  In  fact,  as 
shown  in  Table  2,  the  maximum  profit  on  the  red  program  has  decreased 
from  $51.6  million  to  $28.0  million  while  the  maximum  contractor  loss 
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has  been  reduced  from  $148.4  million  to  $25.3  million.  In  addition,  the 
risk-driven  contract  offers  the  red  program  a  higher  expected  profit,  $9.5 
million  as  compared  to  the  $7.5  million  offered  by  the  FPIF  contract. 
Thus,  contractors  should  clearly  favor  similarly  structured  risk-driven 
contracts  over  the  FPIF  contracts  for  EMD  efforts. 


TABLE  2.  COMPARISON  OF  FPIF  AND  RISK-DRIVEN  CONTRACT 
PROFITS/LOSSES 


FPIF 

Risk-Driven 

Blue 

Red 

Blue  &  Red 

Expected 

Profit 

$10. 9M 

$7.5M 

$9.5M 

Max  Profit  (pO) 

$37.3M 

$51. 6M 

$28. OM 

Max  Loss 

$43.1M 

$148. 4M 

$25. 3M 

A  second  major  observation  from  Figure  7  is  the  flattening  of  the 
sharing  curve  as  the  cost  uncertainty  increases.  Indeed,  it  is  appropriate 
for  the  government  to  share  a  larger  portion  of  the  cost  risk  for  requiring 
greater  innovation.  However,  this  natural  flattening  trend  also  leads  to  a 
potential  drawback  of  the  risk-driven  contract.  As  the  cost  uncertainty 
increases,  the  government  is  forced  to  allocate  more  funding  to  the 
program.  In  the  case  of  the  red  program,  the  government  would  have  to 
allocate  $243.1  million  to  cover  its  share  of  the  contract  to  the  99  per¬ 
cent  confidence  level  without  violating  the  anti-deficiency  laws  (which 
require  the  government  to  budget  to  its  full  contract  liability).  The  gov¬ 
ernment’s  liability  could  be  reduced  to  a  more  reasonable  $174.5  million 
by  agreeing  to  terminate  the  contract  at  the  95  percent  confidence  level. 
However,  the  contractor’s  maximum  liability  would  also  be  reduced  from 
$25.3  million  to  $20.0  million.  Thus,  care  must  be  taken  to  maintain  the 
contractor’s  liability  at  a  sufficient  level  to  still  motivate  unbiased  cost 
estimates. 

Risk-Driven  Contract  Scenario 

The  extra  funding  required  to  cover  the  upper  end  of  the  risk-driven 
contract  value  could  be  considered  the  usual  cost  of  overruns.  Rather 
than  unknowingly  starting  a  system  development  effort  with  an  opti¬ 
mistic  cost  estimate  and  later  dealing  with  an  overrun,  the  risk-driven 
contract  structure  should  bring  more  realism  to  the  initial  affordability 
assessment.  For  example,  consider  the  following  scenario:  Two  contrac- 
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tors  bid  $1.9  billion  and  $2.0  billion  for  a  competitive  cost-plus  EMD 
contract.  The  government’s  independent  cost  estimate  is  $2.5  billion, 
so  the  government  awards  the  $1.9  billion  proposal  and  sets  aside  an 
additional  $400  million  for  management  reserve.  However,  2  years  into 
the  3-year  contract,  the  winning  contractor  projects  an  estimate  at 
completion  of  $3.0  billion.  The  government  is  left  with  two  undesirable 
choices:  cancel  the  program  and  lose  the  investment  or  scramble  to  find 
an  additional  $700  million  to  cover  the  overrun. 

The  scenario  just  described  could  be  improved  through  risk-driven 
contracting.  Being  exposed  to  the  risk  of  a  loss,  the  contractors  should 
provide  more  realistic  cost  proposals.  Perhaps  they  bid  expected  costs 
of  $3.0  billion  and  $3.2  billion.  Even  more,  the  cost  proposals  are  proba¬ 
bilistic,  giving  the  government  much  more  visibility  into  the  range  of 
possible  costs  as  opposed  to  the  point  estimates  normally  provided  today. 
Given  its  $2.5  billion  independent  cost  estimate,  the  government  may 
be  surprised  by  the  high  contractor  cost  estimates  and  needs  to  decide 
whether  the  weapon  system  is  still  worth  the  expected  cost.  However,  in 
this  case,  the  knowledge-based  affordability  assessment  is  made  before 
the  contract  is  started.  And  if  the  contract  is  still  awarded,  there  is  a 
much  better  chance  it  will  be  adequately  funded. 

Conclusion  and  Recommendations 

Risk-driven  contracts  are  aimed  at  reducing  cost  overruns  during 
the  EMD  phase  of  the  defense  acquisition  life  cycle.  Unlike  the  tradi¬ 
tional  cost-plus  contracts  typically  used  during  this  phase,  risk-driven 
contracts  offer  a  structured  approach  to  impose  a  potential  loss  on 
contractors  despite  the  higher  technical  uncertainty.  By  exposing  con¬ 
tractors  to  more  cost  risk,  risk-driven  contracts  should  overcome  the 
issues  related  to  moral  hazard  and  adverse  selection,  and  thus  motivate 
contractors  to  provide  more  realistic  cost  estimates  and  implement  more 
cost  control  discipline  during  contract  execution.  Furthermore,  unlike 
fixed-price  contracts  where  losses  are  unconstrained,  risk-driven  con¬ 
tracts  appropriately  limit  potential  losses,  so  competition  should  not  be 
unduly  hindered. 
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Engineering  Change  Proposals 

To  make  up  for  unrealistic  initial  estimates,  contractors  often  count 
on  ECPs  to  increase  profit  margins.  Unfortunately,  risk-driven  contracts 
do  not  directly  solve  this  dilemma.  However,  with  increased  exposure  to 
losses  on  the  base  contract,  contractors  will  likely: 

•  demand  more  clearly  defined  requirements  and  responsibly 
limit  requirements  creep; 

•  augment  precontract  planning  tasks  (such  as  securing  vendor 
commitments  and  investing  in  technical  feasibility  assessments); 

•  propose  more  mature  technologies  to  reduce  technical 
uncertainty;  and 

•  recommend  incremental  or  spiral  development  strategies. 

While  these  initiatives  may  help  limit  the  need  for  downstream 
changes,  the  government  often  adds  new  contract  requirements  to  keep 
pace  with  commercial  technology  development  or  evolving  warfighter 
needs.  In  this  case,  the  government  should  consider  applying  ECPs  to 
separate  contract  line  items  to  avoid  disrupting  the  base  contract  incen¬ 
tive  structure.  In  addition,  the  government  may  want  to  prenegotiate  use 
of  the  original  probabilistic  sharing  structure  for  all  ECPs  to  streamline 
future  contract  actions. 


Risk-driven  contracts  should  also  help  limit  the  government’s  ability 
to  commit  to  too  many  programs  by  fostering  knowledge-based  afford¬ 
ability  assessments.  By  requiring  the  government  to  set  aside  funding  to 
cover  the  entire  contract  liability,  the  anti-deficiency  laws  should  help 
reduce  overextended  budgets  and  the  funding  instability  they  induce. 
The  government  still  reserves  the  right  to  deobligate  funding  from  a 
risk-driven  contract  in  response  to  changing  priorities.  However,  upset¬ 
ting  the  risk- driven  sharing  ratios  will  require  more  negotiation  effort 
than,  for  example,  borrowing  money  from  a  CPAF  contract.  This  higher 
negotiation  threshold  may  provide  risk- driven  contracts  slightly  more 
protection  from  funding  cuts  and  the  resultant  schedule  delays. 
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In  implementing  the  Weapon  Systems  Acquisition  Reform  Act  of 
2009,  the  USD(AT&L)  directed  program  cost  estimates  to  be  stated  at 
the  80  percent  confidence  level  (Carter,  2009,  p.  6).  However,  this  direc¬ 
tive  only  applies  to  Office  of  the  Secretary  of  Defense  and  Service  cost 
estimates,  and  not  contractor  proposals,  which  normally  provide  no 
stated  confidence  level  for  their  point  estimates.  To  enable  risk-driven 
contracts,  the  government  needs  to  start  requiring  probabilistic  cost 
estimates  as  part  of  its  Request  for  Proposal  instructions.  Surprisingly, 
this  is  not  already  common  practice,  and  the  government  continues  to 
make  huge  financial  commitments  without  soliciting  the  confidence  level 
of  contractor  cost  estimates. 

Weitzman  (1980)  states,  “The  government  is  frequently  assumed 
to  be  risk-neutral  as  a  first  approximation”  (p.  723).  Thus,  in  evaluating 
probabilistic  cost  estimates,  a  risk-neutral  program  office  should  gener¬ 
ally  select  the  proposal  with  the  lowest  expected  cost  (all  other  factors 
being  equal).  However,  given  the  current  fiscal  environment  and  the 
negative  perception  caused  by  overruns,  a  risk- averse  program  office  may 
want  to  also  consider  the  variance  of  each  cost  estimate.  In  other  words, 
it  may  be  prudent  to  select  a  proposal  with  a  higher  expected  cost  if  it  has 
a  lower  maximum  liability  than  the  other  options. 
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EndNotes 

1  For  practical  purposes,  the  expected  profit  calculations  were  cut  off  at  the  99 
percent  confidence  levels  because  the  100  percent  confidence  levels  theoretically 
extend  to  infinity. 

2  The  Nobel  Prize  is  not  awarded  posthumously;  otherwise,  it  is  generally  regarded  as  a 
given  that  Tversky  would  have  shared  the  honor. 
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